<!-------- @HEADER
 !
 ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 !
 !  Zoltan Toolkit for Load-balancing, Partitioning, Ordering and Coloring
 !                  Copyright 2012 Sandia Corporation
 !
 ! Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation,
 ! the U.S. Government retains certain rights in this software.
 !
 ! Redistribution and use in source and binary forms, with or without
 ! modification, are permitted provided that the following conditions are
 ! met:
 !
 ! 1. Redistributions of source code must retain the above copyright
 ! notice, this list of conditions and the following disclaimer.
 !
 ! 2. Redistributions in binary form must reproduce the above copyright
 ! notice, this list of conditions and the following disclaimer in the
 ! documentation and/or other materials provided with the distribution.
 !
 ! 3. Neither the name of the Corporation nor the names of the
 ! contributors may be used to endorse or promote products derived from
 ! this software without specific prior written permission.
 !
 ! THIS SOFTWARE IS PROVIDED BY SANDIA CORPORATION "AS IS" AND ANY
 ! EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 ! IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 ! PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL SANDIA CORPORATION OR THE
 ! CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 ! EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 ! PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 ! PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 ! LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 ! NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 ! SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 !
 ! Questions? Contact Karen Devine	kddevin@sandia.gov
 !                    Erik Boman	egboman@sandia.gov
 !
 ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 !
 ! @HEADER
-------> 

<HTML>
<HEAD>
   <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
   <META NAME="GENERATOR" CONTENT="Mozilla/4.04 [en] (X11; U; SunOS 4.1.3_U1 sun4m) [Netscape]">
   <META NAME="sandia.approved" CONTENT="SAND99-1376">
   <META NAME="author" CONTENT="karen devine, kddevin@sandia.gov">

   <TITLE> Zoltan Developer's Guide:  Parallel Routines</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF">

<div ALIGN=right><b><i><a href="dev.html">Zoltan Developer's Guide</a>&nbsp; |&nbsp; <a href="dev_services_objlist.html">Next</a>&nbsp; |&nbsp; <a href="dev_services_params.html">Previous</a></i></b></div>
<!------------------------------------------------------------------------->


<H2>
<A NAME="parallel_compute"></A>Parallel Computing Routines</H2>
Parallel computing utilities are described in this section.
<BLOCKQUOTE><B><A HREF="#Zoltan_Print_Sync_Start">Zoltan_Print_Sync_Start</A></B>
/ <B><A HREF="#Zoltan_Print_Sync_End">Zoltan_Print_Sync_End</A></B>:&nbsp; provide
synchronization of processors for I/O (with <A HREF="#print_sync_example">example</A>).
<BR><B><A HREF="#Zoltan_Print_Stats">Zoltan_Print_Stats</A></B> : print statistics
about a parallel variable.</BLOCKQUOTE>

<!------------------------------------------------------------------------->
<hr>
<A NAME="Zoltan_Print_Sync_Start"></A>
<HR>
void <B>Zoltan_Print_Sync_Start</B>(MPI_Comm <i>communicator</i>,
int <I>do_print_line</I>);&nbsp;
<HR>

<P>The <B>Zoltan_Print_Sync_Start</B> function is adapted from work of John
Shadid for the MPSalsa project at Sandia National Laboratories. With <B><A HREF="#Zoltan_Print_Sync_End">Zoltan_Print_Sync_End</A></B>,
it provides synchronization so that one processor in the Zoltan
communicator can complete its I/O before the next processor begins its
I/O. This synchronization utility is useful for debugging algorithms, as
it allows the output from processors to be produced in an organized manner.
It is, however, a serializing process, and thus, does not scale well to
large number of processors.

<P><B>Zoltan_Print_Sync_Start</B> should called by each processor in the MPI
communicator before the desired I/O is performed. <B><A HREF="#Zoltan_Print_Sync_End">Zoltan_Print_Sync_End</A></B>
is called by each processor after the I/O is performed. No communication
can be performed between calls to <B>Zoltan_Print_Sync_Start</B> and <B><A HREF="#Zoltan_Print_Sync_End">Zoltan_Print_Sync_End</A></B>.
See the <A HREF="#print_sync_example">example</A> below for usage of <B>Zoltan_Print_Sync_Start</B>.
<BR>&nbsp;
<TABLE WIDTH="100%" >
<TR VALIGN=TOP>
<TD VALIGN=TOP WIDTH="20%"><B>Arguments:</B></TD>

<TD WIDTH="80%"></TD>
</TR>

<TR>
<TD VALIGN=TOP>&nbsp;&nbsp;&nbsp; <I>communicator</I></TD>

<TD>The MPI communicator containing all processors to participate in the
synchronization.
</TD>
</TR>

<TR>
<TD VALIGN=TOP>&nbsp;&nbsp;&nbsp; <I>do_print_line</I></TD>

<TD>A flag indicating whether to print a line of "#" characters before
and after the synchronization block. If <I>do_print_line</I> is TRUE, a
line is printed; no line is printed otherwise.</TD>
</TR>
</TABLE>
&nbsp;

<P>

<!------------------------------------------------------------------------->
<hr>
<A NAME="Zoltan_Print_Sync_End"></A>
<HR>
void <B>Zoltan_Print_Sync_End</B>(MPI_Comm <i>communicator</i>,
int <I>do_print_line</I>);&nbsp;
<HR>

<P>The <B>Zoltan_Print_Sync_End </B>function is adapted from work of John Shadid
for the MPSalsa project at Sandia National Laboratories. With <B><A HREF="#Zoltan_Print_Sync_Start">Zoltan_Print_Sync_Start</A></B>,
it provides synchronization so that one processor in the Zoltan
communicator can complete its I/O before the next processor begins its
I/O. This synchronization utility is useful for debugging algorithms, as
it allows the output from processors to be produced in an organized manner.
It is, however, a serializing process, and thus, does not scale well to
large number of processors.

<P><B><A HREF="#Zoltan_Print_Sync_Start">Zoltan_Print_Sync_Start</A></B> should
called by each processor in the MPI communicator before the desired I/O
is performed. <B>Zoltan_Print_Sync_End</B> is called by each processor after
the I/O is performed. No communication can be performed between calls to
<B><A HREF="#Zoltan_Print_Sync_Start">Zoltan_Print_Sync_Start</A></B> and <B>Zoltan_Print_Sync_End</B>.
See the <A HREF="#print_sync_example">example</A> below for usage of <B>Zoltan_Print_Sync_End</B>.
<BR>&nbsp;
<TABLE WIDTH="100%" >
<TR VALIGN=TOP>
<TD VALIGN=TOP WIDTH="20%"><B>Arguments:</B></TD>

<TD WIDTH="80%"></TD>
</TR>

<TR>
<TD VALIGN=TOP>&nbsp;&nbsp;&nbsp; <I>communicator</I></TD>

<TD>The MPI communicator containing all processors to participate in the 
synchronization.
</TD>
</TR>

<TR>
<TD VALIGN=TOP>&nbsp;&nbsp;&nbsp; <I>do_print_line</I></TD>

<TD>A flag indicating whether to print a line of "#" characters before
and after the synchronization block. If <I>do_print_line</I> is TRUE, a
line is printed; no line is printed otherwise.</TD>
</TR>
</TABLE>

<!------------------------------------------------------------------------->
<hr>
<A NAME="Zoltan_Print_Stats"></A>
<HR WIDTH="100%">
void <B>Zoltan_Print_Stats</B>(MPI_Comm <i>comm</i>, int <i>debug_proc</i>,
double <i>x</i>, char <i>*msg</i>);
<BR>
<HR WIDTH="100%">
<BR>Zoltan_Print_Stats is a very simple routine that computes the maximum and
sum of the variable <I>x</I> over all processors associated with the MPI
communicator <i>comm</i>.
It also computes and prints the imbalance
of <I>x</I>, that is, the maximum value divided by the average minus one.
If <I>x </I>has the same value on all processors, the imbalance is zero.
<p>
<TABLE BORDER=0 WIDTH="100%" NOSAVE >
<tr>
<TD VALIGN=TOP WIDTH="20%"><B>Arguments:</B></TD>
<td WIDTH="80%"></td>
</tr>
<TR>
<TD>&nbsp;&nbsp;&nbsp; <I>comm&nbsp;</I></TD>

<TD>The MPI Communicator to be used in maximum and sum operations.
</TD>
</TR>
<TR>
<TD>&nbsp;&nbsp;&nbsp; <I>debug_proc&nbsp;</I></TD>

<TD>The processor from which output should be printed.
</TD>
</TR>


<TR>
<TD>&nbsp;&nbsp;&nbsp; <I>x</I></TD>

<TD>The variable of which one wishes to display statistics.</TD>
</TR>

<TR NOSAVE>
<TD>&nbsp;&nbsp;&nbsp; <I>msg</I></TD>

<TD NOSAVE>A string that typically describes the meaning of <I>x.</I></TD>
</TR>
</TABLE>

<HR WIDTH="100%">
<H4>

<!------------------------------------------------------------------------->

<HR WIDTH="100%"><A NAME="print_sync_example"></A><B>Example Using Zoltan_Print_Sync_Start/Zoltan_Print_Sync_End</B></H4>
&nbsp;
<CENTER><TABLE BORDER=2 COLS=1 WIDTH="90%" NOSAVE >
<TR>
<TD><TT>...</TT>&nbsp;
<BR><TT>if (zz->Debug_Level >= ZOLTAN_DEBUG_ALL) {</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp; int i;</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp; Zoltan_Print_Sync_Start(zz->Communicator, TRUE);</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp; printf("Zoltan: Objects to be exported from Proc
%d\n", zz->Proc);</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp; for (i = 0; i &lt; *num_export_objs; i++) {</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf("&nbsp;&nbsp;&nbsp;
Obj: ");</tt>
<BR><TT>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ZOLTAN_PRINT_GID(zz, &((*export_global_ids)[i*zz->Num_GID]));
<BR><TT>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; printf(" Destination: %4d\n",</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(*export_procs)[i]);</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp; }</TT>&nbsp;
<BR><TT>&nbsp;&nbsp;&nbsp; Zoltan_Print_Sync_End(zz->Communicator, TRUE);</TT>&nbsp;
<BR><TT>}</TT></TD>
</TR>

<CAPTION ALIGN=BOTTOM><I>Example usage of Zoltan_Print_Sync_Start and Zoltan_Print_Sync_End
to synchronize output among processors.&nbsp; (Taken from Zoltan_LB_Partition in
lb/lb_balance.c.)</I></CAPTION>
</TABLE></CENTER>
&nbsp;

<P>
<HR WIDTH="100%">
<BR>[<A HREF="dev.html">Table of Contents</A>&nbsp; |&nbsp; <A HREF="dev_services_objlist.html">Next:&nbsp;
Object List Function</A>&nbsp; |&nbsp; <A HREF="dev_services_params.html">Previous:&nbsp;
Parameter Setting Routines</A>&nbsp; |&nbsp; <a href="https://www.sandia.gov/general/privacy-security/index.html">Privacy and Security</a>]
</BODY>
</HTML>
